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ABSTRACT 



Three analyses were conducted for this progress report of 
the Reading Excellence Act as implemented in Alabama public schools. The 
first analysis involved third grade reading comprehension scores over the 
3-year period (1999-2001). Overall, reading comprehension scores remained 
stable. Based on multiple criteria, schools that either failed to meet 
expectations or exceeded them were identified. The second analysis examined 
changes from second to third grade for a matched sample of students 
participating REA schools. For all schools, the scores were higher in second 
grade. This may be, in part, due to the nature of the evaluation instrument 
used, the Alabama Early Learning Inventory. As in the first analysis, schools 
that either failed to meet expectations or exceeded them were identified. The 
third analysis examined relationships among the scores from the kindergarten, 
first, and second grade assessments with Stanford Achievement Test, Level 9 
scores in third grade. Results revealed strong, positive relationships 
between each of the second grade strands and third grade performance. (RS) 
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Analysis #1 

Analysis of Third Grade Reading Comprehension (1999-2001) . 

Third grade SAT9 Reading Comprehension scores were examined over a three-year 
period (1999-2001). While the scores at each year are attributed to different students, this 
analysis provides some basis for understanding the changes in achievement at the third 
grade level in participating REA schools. For the purposes of this examination, schools 
were categorized into four groups: 1) Non-REA schools, 2) REA schools implementing 
tutorial assistance (TA), 3) REA Schools implementing local reading improvement (LRI), 
4) REA schools employing both TA and LRI. Descriptive information for each of these 
four groups of schools is summarized below. 
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Grade 3 Reading Comprehension - 1999-2001 



REATYPE 


Mean 


Std. Deviation 


Number of 
Schools 


Reading Comp 1999 


Non-REA Schools 


49.5039 


7.6422 


668 




Tutorial Assistance 


39.9013 


3.1498 


16 




Local Reading 
Improvement 


39.9324 


4.3625 


39 




Both TA and LRI 


44.2520 


5.9123 


7 




Total 


48.7317 


7.8430 


730 


Reading Comp 2000 


Non-REA Schools 


49.5813 


7.5027 


668 




Tutorial Assistance 


40.9260 


4.8604 


16 




Local Reading 
Improvement 


40.4242 


4.7016 


39 




Both TA and LRI 


40.8136 


5.8047 


7 




Total 


48.8183 


7.7268 


730 


Reading Comp 2001 


Non-REA Schools 


49.3073 


7.9993 


668 




Tutorial Assistance 


42.9933 


3.7819 


16 




Local Reading 
Improvement 


39.8988 


4.8385 


39 




Both TA and LRI 


42.5105 


2.8534 


7 




Total 


48.6011 


8.1045 


730 



Overall, 3 rd grade reading comprehension scores remained stable from 1999-2001, 
increasing slightly from 48.73 in 1999 to 48.82 in 2000 before falling dropping to 48.60 
in 2001. The profile of non-REA schools is very similar, increasing from 1999-2000 and 
falling slightly in 2001. The profiles of REA schools from 1999-2001, however, are 
different. 



While REA schools implementing tutorial assistance have gradually increased their third 
grade reading comprehension each year (39.90 in 1999 to 42.99 in 2001), the other two 
groups have experienced different trends from 1999-2001. REA-LRI schools 
experienced an increase from 1999 to 2000 (39.93 to 40.42) before returning to their 
1999 level this past year (39.90). The seven schools using both TA and LRI declined 
from 44.25 in 1999 to 40.81 in 2000 before an increase to 42.51 this past year. The 
profiles for each group of schools are displayed in the Figure below. 
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Third Grade Reading Comprehension 
1999-2001 




Year of Test 
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Regression Analysis (Using Grade 3 Cohorts) 



Regression analysis was used to identify academic and demographic predictors of third- 
grade SAT-9 Reading comprehension. The two academic variables used were the 
previous two years SAT-9 scores. The demographic predictors included percentages of: 

1) students receiving a free/reduced lunch, 2) minority students, 3) special education 
students, 4) female students, and 5) attendance. These regressions were used to predict 
school performance based on their academic and demographic situation and determine 
the extent to which their actual performance met these predicted expectations. A school’s 
departure from expected performance was then used as one criterion forjudging the 
success of the school over the past year. 



Summary of Regression Analysis 
Third Grade Reading Comprehension 






Academic Predictors 


R= .845, R 2 = .714. p < .001 


RC 2000 


492 *** 


RC 1999 


392 *** 


Demographic Predictors 


R= .780, R 2 = .608. p < .001 


Percent Free/reduced lunch 


- 484 *** 


Percent Minority 


-.365 *** 


Attendance Rate 


.157 *** 


Percent Female 


.053 * 


Percent Special Education 


-.002 


* p < .05, ** p < .01, *** p < .001 





The overall relationship between the previous two years of SAT9 scores was very high 
(R= .845). The most recent reading comprehension performance (RC2000) was 
identified as the most contributing predictor of current performance. The relationship 
between demographic variables and current SAT9 scores in reading comprehension was 
slightly lower, but very strong (R= .780). Among the demographic variables considered, 
percent of free/reduced lunch and percent of minority students related negatively to 
reading comprehension while attendance rate and percent of female students correlated 
positively. 
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Selection of Outlier Schools (Using Grade 3 Cohorts) 

Based on multiple criteria, schools were identified as outliers in that they either failed to 
meet expectations or exceeded them. The following criteria were used to select outlier 
schools. 

1. Change in SAT9 Reading Comprehension Scores from 2000 to 2001, using grade 
3 performance. To judge this criterion, gains from year to year were transformed 
to Z scores (standardized values) so that such gains or losses could be viewed in 
reference to the average gain or loss for all third grades. 

2. Comparison of current performance (RC 2001) to what is expected based on the 
past two years scores. To judge this criterion, 2001 scores were predicted form the 
school’s academic history to arrive at an expected level of performance. The 
difference between each school’s actual performance and what would be expected 
was standardized (using Z scores) so that the discrepancy could be examined 
more easily. 

3. Comparison of current performance (RC 2001) to what would be expected given 
their demographic situation. This criterion was judged much like #2 above, using 
an expected performance based on demographic predictors. 

Based on the analysis of matched-cases school-wide performance using the above 

criteria, I would select the following REA schools as positive and negative outliers 

Positive Outliers: 

The following schools experienced the largest increases in third-grade reading 
comprehension over the past year. These schools have also exceeded demographic and 
academic expectations in 2001. 

F. S. Erwin Elementary (Wilcox County) 

Shiloh Elementary (Dallas County) 

North Sumter Junior High (Sumter County) 

Elyton Elementary (Birmingham City) 

Glendale (Mobile County) 

Gaston (Birmingham City) 

Peter Joe Hamilton (Mobile County) 

Oscar W. Adams (Gadsden City) 

Negative Outliers 

The following schools experienced the largest declines in third-grade reading 
comprehension over the past year. These schools have also fallen short of demographic 
and academic expectations in 2001. 

Lisman Jr. High (Choctaw County) 

Monroe Sr. High (Monroe County) 

Abrams Elementary (Bessesmer City) 

Southern Choctaw Elementary (Choctaw County) 

Dunbar Elementary (Montgomery County) 

Alberta Elementary (Tuscaloosa City) 
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Analysis #2 

Analysis of Change from Grade 2 to Grade 3 in REA Schools 

The second analysis examined changes from grades 2 to 3 for a matched sample of 
students in participating REA schools. The SAT9 is administered in grades 3 through 11. 
Therefore, to assess change from grade 2 to grade 3, an alternative assessment was used 
for grade 2. The Alabama Early Learning Inventory (AELI) is a state test designed for 
Kindergarten through grade 2. . The AELI was administered in the falls of 1999 and 
2000. 

The Kindergarten tests includes sections pertaining to phonemic awareness, concepts of 
print, expressive language and graphophonemic knowledge. The first grade tests also 
include sections regarding phonemic awareness, concepts of print and graphophonemic 
knowledge. In addition, the grade 1 test examines passage comprehension and 
conventions of writing. Finally, the second grade test examines the areas of passage 
comprehension and oral reading 

Passage comprehension is most closely aligned with reading comprehension, which is a 
primary focus of the SAT9 in third grade. Therefore, the subtest of passage 
comprehension subtest, administered to 2 nd graders in the fall of 1999, was used as a 
baseline measure. These second graders became third graders in the fall of 2000, the first 
year of participation in REA and completed the SAT subtest of reading comprehension in 
the spring of 2001. 

For the purposes of this analysis, schools were categorized into four groups: 1) Non-REA 
schools, 2) REA schools implementing tutorial assistance (TA), 3) REA Schools 
implementing local reading improvement (LRI), 4) REA schools employing both TA and 
LRI. Descriptive information for each of these four groups of schools is summarized 
below. 



Change from Grade 2 to Grade 3 











Number of 






Mean 


Std. Deviation 


Schools 


Passage Comp - Grade 2 


Non-REA Schools 


51.1567 


7.7109 


701 




Tutioral Assistance 


44.3044 


4.0130 


16 




Local Reading 
Improvement 


43.1239 


5.4888 


40 




Both 


45.5563 


6.9363 


7 




Total 


50.5413 


7.8141 


764 


Reading Comp - Grade 3 


Non-REA Schools 


48.8091 


7.9682 


701 




Tutioral Assistance 


42.1089 


3.2587 


16 




Local Reading 
Improvement 


39.9548 


5.0827 


40 




Both 


43.1441 


4.3315 


7 




Total 


48.1533 


8.0524 


764 
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For all schools, the scores were higher in second grade. This may be, in part, due to the 
nature of the test. The second grade assessment norms are based solely on the state’s 
performance while the SAT9 is normed using a national sample. The profiles of each 
group are similar, as displayed in the figure below. 



Change from Grade 2 to Grade 3 




Grade Level 



Regression Analysis (Using Grade 3 Cohorts) 



Regression analysis was used to identify academic and demographic predictors of third- 
grade SAT-9 Reading comprehension. The academic variable used were the previous 
year’s performance in Grade 2 - Passage Comprehension and Oral Reading. The 
demographic predictors included percentages of: 1) students receiving a ffee/reduced 
lunch, 2) minority students, 3) special education students, 4) female students, and 5) 
attendance. These regressions were used to predict school performance based on their 
academic and demographic situation and determine the extent to which their actual 
performance met these predicted expectations. A school’s departure from expected 
performance was then used as one criterion forjudging the success of the school over the 
past year. 



Summary of Regression Analysis 
Third Grade Reading Comprehension 






Academic Predictors 


R= .793, R 2 = .629. p < .001 


Grade 2 Passage Comp 


.663 *** 


Grade 2 — Oral Reading 


.160 *** 


Demographic Predictors 


R= .770, R 2 = .593. p < .001 


Percent Free/reduced lunch 


- 487 *** 


Percent Minority 


..354 *** 


Attendance Rate 


.165 *** 


Percent Female 


.032 


Percent Special Education 


-.017 


* p < .05, ** p < .01, *** p < .001 





The overall relationship between the previous two years of SAT9 scores was very high 
(R= .793) with Grade 2 - Passage Comprehension identified as the most contributing 
predictor of current performance. The relationship between demographic variables and 
current SAT9 scores in reading comprehension was slightly lower, but very strong (R= 
.770). Among the demographic variables considered, percent of free/reduced lunch and 
percent of minority students related negatively to reading comprehension while 
attendance rate correlated positively. 
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Selection of Outlier Schools (Using Grade 3 Cohorts) 

Based on multiple criteria, schools were identified as outliers in that they either failed to 
meet expectations or exceeded them. The following criteria were used to select outlier 
schools. 

1. Change from Grade 2 to Grade 3, using matched cases. To judge this criterion, 
gains were transformed to Z scores (standardized values) so that such gains or 
losses could be viewed in reference to the average gain or loss for all schools. 

2. Comparison of third grade performance (RC 2001) to what is expected based on 
the grade2 performance. To judge this criterion, 2001 scores were predicted form 
the school’s academic history to arrive at an expected level of performance. The 
difference between each school’s actual performance and what would be expected 
was standardized (using Z scores) so that the discrepancy could be examined 
more easily. 

3. Comparison of current performance (RC 2001) to what would be expected given 
their demographic situation. This criterion was judged much like #2 above, using 
an expected performance based on demographic predictors. 

Based on the analysis of matched-cases school-wide performance using the above 
criteria, I would select the following REA schools as positive and negative outliers 

Positive Outliers: 

The following schools experienced the largest increases from 2 nd to 3 rd grade over the 
past year. These schools have also exceeded demographic and academic expectations. 

Gaston (Birmingham City) 

W.E. Striplin (Gadsden City) 

W.H. Brazier (Mobile County) 

F. S. Erwin Elementary (Wilcox County) 

Oscar W. Adams (Gadsden City) 

Martha Thomas (Mobile County) 

Monroe Sr. High (Monroe County) 

Negative Outliers 

The following schools experienced the largest declines from 2 nd to 3 rd grade over the past 
year. These schools have also fallen short of demographic and academic expectations. 

York West End Jr. High (Sumter County) 

Livingston Jr. High (Sumter County) 

Fruitdale High (Washington County) 

Southlawn Elementary ((Montgomery County) 

Donald Elementary (Fairfield County) 

Monroeville Elementary (Monroe County) 

Stillman Heights (Tuscaloosa City) 

Alberta Elementary (Tuscaloosa City) 

T.S. Morris (Montgomery County) 
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The following schools were identified as positive outliers using both analysis procedures: 

Gaston (Birmingham City) 

F. S. Erwin Elementary (Wilcox County) 

Oscar W. Adams (Gadsden City) 



The following school was identified as negative outliers using both analysis procedures. 
Alberta Elementary (Tuscaloosa City) 



Analysis # 3 

Relationship Between K,l,2 Assessments and SAT Scores 

The third analysis examined relationships among the scores from the K,l, and 2 
assessments with SAT scores in third grade. 

Relationships Between Grade 2 and Grade 3 

The bivariate relationships between each of the second grade strand scores and 
third grade SAT scores are summarized in the table below. Each of the strands and SAT 
scores are described below: 

Grade 2 

Strand 1 - Passage Comprehension, Strand 2 - Oral Reading, Strand 3 - Total 
Grade 3 

Reading Comprehension, Vocabulary, Total Reading 



Correlations Between Grade 2 (Strands 1,2,3) and Grade 3 
SAT Reading Scores 







5 


CM 


CO 






Q 


D 






i 


i 


i 






cz 

i— 


cz 

h- 


F 






CO 


CO 


CO 


Read 


Pearson Correlation 


.616** 


.461** 


.612* 1 


Comp 


Sig. (2-tailed) 


.000 


.000 


.000 




N 


44267 


44299 


44088 


Vocab 


Pearson Correlation 


.587** 


474 ** 


.600* 1 




Sig. (2-tailed) 


.000 


.000 


.000 




N 


44267 


44299 


44088 


Total 


Pearson Correlation 


.630** 


484** 


.632* 1 


Read 


Sig. (2-tailed) 


.000 


.000 


.000 




N 


44267 


44299 


44088 



“• Correlation is significant at the 0.01 level (2-tailed). 



These results reveal strong, positive relationships between ach of the 2 nd grade strands 
and third grade performance. Whereas strand one, passage comprehension, correlates 
more strongly with reading comprehension (r=.616), stand two, oral reading, resulted in a 
stronger relationship with third grade vocabulary (r=.474). 
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